Scoring of Response Based on Suitability of Dialogue-act and Content Similarity

We present an approach to scoring candidate utterances in a large repository of short text conversation (STC) data to select those to be used as a suitable response to a newly given utterance. Candidate utterances are evaluated based on the suitability of a dialogue-act and the content similarity. The estimation of the suitability of a dialogue-act is implemented by learning the trend of a dialogue-act pair that frequently appears in the repository. Also, we calculated the content similarity between utterances by means of the cosine similarity of topic vectors using LDA and IDF. By multiplying these values, those candidates which are suitable in terms of function and content attain a high score. As a result of the experimental evaluation, for content similarity, it was found that increasing the weighting of the IDF produces a better accuracy.